Lustre: A Scalable, High-Performance File System
ثبت نشده
چکیده
Today's network-oriented computing environments require high-performance, network-aware file systems that can satisfy both the data storage requirements of individual systems and the data sharing requirements of workgroups and clusters of cooperative systems. The Lustre File System, an open source, high-performance file system from Cluster File Systems, Inc., is a distributed file system that eliminates the performance, availability, and scalability problems that are present in many traditional distributed file systems. Lustre is a highly modular next generation storage architecture that combines established, open standards, the Linux operating system, and innovative protocols into a reliable, network-neutral data storage and retrieval solution. Lustre provides high I/O throughput in clusters and shared-data environments and also provides independence from the location of data on the physical storage, protection from single points of failure, and fast recovery from cluster reconfiguration and server or network outages.
منابع مشابه
Tuning HDF5 for Lustre File Systems
HDF5 is a cross-platform parallel I/O library that is used by a wide variety of HPC applications for the flexibility of its hierarchical object-database representation of scientific data. We describe our recent work to optimize the performance of the HDF5 and MPI-IO libraries for the Lustre parallel file system. We selected three different HPC applications to represent the diverse range of I/O ...
متن کاملA Scalable Message Passing Interface Implementation of an Ad-Hoc Parallel I/o system
In this paper we present the novel design, implementation, and evaluation of an ad-hoc parallel I/O system (AHPIOS). AHPIOS is the first scalable parallel I/O system completely implemented in the Message Passing Interface (MPI). The MPI implementation brings the advantages of portability, scalability and high performance. AHPIOS allows MPI applications to dynamically manage and scale distribute...
متن کاملPersonalized Cloud Storage System: A Combination of LDAP Distributed File System
“Cloud computing” gradually flourish, a wide range of distributed storage systems are increasingly diverse, Like of Gluster, Ceph, Lustre, as well as Hadoop, etc.. In this paper, we propose a personal cloud storage system Integrated with pNFS, it can be accessed in parallel for scalable performance. Besides, data backup and failover mechanism are designed. We expect that the function of the pro...
متن کاملTowards a High Performance Implementation of MPI-IO on the Lustre File System
Abs tra ct—Lustre is becoming an increasingly important file system for large-scale computing clusters. The problem is that many dataintensive applications use MPI-IO for their I/O requirements, and it has been well documented that MPI-IO performs poorly in a Lustre file system environment. However, the reasons for such poor performance are not currently well understood. We believe that the pri...
متن کاملInfluence of File Systems on Performance When Working with an Abundance of Small Files
High-performance computing is widely used within the scientific community to perform demanding computational work. Using the resources available at a high-performance center in an efficient manner is of great importance. One potential bottleneck for high-performance computing is file systems. In this study two different file systems, the Lustre file system and MATLAB Datastore, have been evalua...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2003